Presently, data prefetchers exist in hardware and software solutions in the form of data prefetch. A linear or stride prefetcher recognizes linear sequences of memory accesses, e.g. loops always requesting the next 8 byte. Typical source code patterns, where this prefetching strategy works successfully, are loops iterating over vectors and arrays. This type of prefetching is even able to prefetch cache lines that were previously never used. Typically, performance may differ by up to 20% wall clock time depending on whether the prefetcher was able to prefetch as intended.
Presently, a prefetch instruction in the code allows for programs to announce future memory accesses. However, problems hereby may include determining the right location in the code to issue a prefetch instruction, the execution time consumption by the prefetch instructions itself, and the overall increase of code size. Also, placement rules may change from processor model to processor model, causing problems with old code already in use at customer sites.
High performance architectures and parallel computing systems may employ list-based prefetching. The prefetch engine keeps track of all current cache misses. Information regarding prior address misses are recorded in a list of addresses. The prefetch engine prefetches data according to the list, if there is a match between the current cache miss address and one of the addresses in the list. There are several refinements to this strategy in place to determine when this strategy should get used and to improve the effectiveness of the prefetching. Source code contains instructions to mark the sections for a type of prefetching to be performed.
Utilization of a library-based prefetch scheme can include prefetching by a helper thread running on a spare processor and included in the data-structure library code. Effective prefetching is possible with the library's knowledge of the data-structure type and frequent access patterns. This solution runs completely in software and requires no adoption of existing program code.